Search results for "Cross entropy"
showing 9 items of 9 documents
Information Decomposition in Bivariate Systems: Theory and Application to Cardiorespiratory Dynamics
2015
In the framework of information dynamics, the temporal evolution of coupled systems can be studied by decomposing the predictive information about an assigned target system into amounts quantifying the information stored inside the system and the information transferred to it. While information storage and transfer are computed through the known self-entropy (SE) and transfer entropy (TE), an alternative decomposition evidences the so-called cross entropy (CE) and conditional SE (cSE), quantifying the cross information and internal information of the target system, respectively. This study presents a thorough evaluation of SE, TE, CE and cSE as quantities related to the causal statistical s…
SCORING ALTERNATIVE FORECAST DISTRIBUTIONS: COMPLETING THE KULLBACK DISTANCE COMPLEX
2018
We develop two surprising new results regarding the use of proper scoring rules for evaluating the predictive quality of two alternative sequential forecast distributions. Both of the proponents prefer to be awarded a score derived from the other's distribution rather than a score awarded on the basis of their own. A Pareto optimal exchange of their scoring outcomes provides the basis for a comparison of forecast quality that is preferred by both forecasters, and also evades a feature of arbitrariness inherent in using the forecasters' own achieved scores. The well-known Kullback divergence, used as a measure of information, is evaluated via the entropies in the two forecast distributions a…
Conditional Entropy-Based Evaluation of Information Dynamics in Physiological Systems
2014
We present a framework for quantifying the dynamics of information in coupled physiological systems based on the notion of conditional entropy (CondEn). First, we revisit some basic concepts of information dynamics, providing definitions of self entropy (SE), cross entropy (CE) and transfer entropy (TE) as measures of information storage and transfer in bivariate systems. We discuss also the generalization to multivariate systems, showing the importance of SE, CE and TE as relevant factors in the decomposition of the system predictive information. Then, we show how all these measures can be expressed in terms of CondEn, and devise accordingly a framework for their data-efficient estimation.…
Cross-entropy-based adaptive optimization of simulation parameters for Markovian-driven service systems
2005
Abstract Markov fluid models represent a general description of the process of service request arrivals to service systems. The solution of performance analysis problems incorporating them often calls for a simulation approach, for which a reference methodology is Importance Sampling. However, in this case the appropriate choice of the biasing conditions is a problem in itself. In this paper an iterative method based on the cross-entropy is proposed for this choice. The equations are given that allow to derive the biasing conditions from the simulation itself. The application of the proposed method to three different sample cases, referring to one transient scenario (finite time horizon and…
Spatial spillovers in France: a study on individual count data at the city level
2007
Our study aims to measure the effects of spatial R&D spillovers on firms' patent production at the city level. We use an original method to estimate the spatial dimension of spillovers using count data. The method, based on a generalized cross entropy approach, allows us to test spatial auto-correlation. The main result is that when there are local spillovers, their impact on knowledge production is different according to the geographical area and the sector.
Hybridizing the cross-entropy method: An application to the max-cut problem
2009
Cross-entropy has been recently proposed as a heuristic method for solving combinatorial optimization problems. We briefly review this methodology and then suggest a hybrid version with the goal of improving its performance. In the context of the well-known max-cut problem, we compare an implementation of the original cross-entropy method with our proposed version. The suggested changes are not particular to the max-cut problem and could be considered for future applications to other combinatorial optimization problems.
Neural Networks with Multidimensional Cross-Entropy Loss Functions
2019
Deep neural networks have emerged as an effective machine learning tool successfully applied for many tasks, such as misinformation detection, natural language processing, image recognition, machine translation, etc. Neural networks are often applied to binary or multi-class classification problems. In these settings, cross-entropy is used as a loss function for neural network training. In this short note, we propose an extension of the concept of cross-entropy, referred to as multidimensional cross-entropy, and its application as a loss function for classification using neural networks. The presented computational experiments on a benchmark dataset suggest that the proposed approaches may …
Investigating cardiac and respiratory determinants of heart rate variability in an information-theoretic framework.
2014
This study was aimed at comparing two alternative information-theoretic approaches for the combined analysis of heart rate variability (HRV) and respiration variability (RV). The approaches decompose the predictive information about HRV in two terms, quantifying respectively the information stored into HRV and that transferred to HRV from RV. Storage and transfer were assessed by the popular self entropy (SE) and transfer entropy (TE) measures, as well as by the alternative conditional SE (cSE) and cross entropy (CE) measures. The comparison was performed at a theoretical level, computing the exact values of the four measures for simulated cardiorespiratory dynamics, and on real data, estim…
Load match optimisation of a residential building case study: A cross-entropy based electricity storage sizing algorithm
2015
Abstract The EU EPBD recast regulation marked the application of the net zero energy building (Net ZEB) concept in all fields of building construction in Europe as a building able to generate as much energy as it consumes over a selected time frame. A more detailed insight is however needed, as even if a building achieves a long-term energy balance between energy generated and consumed, smaller time scales must also be considered. For example, from the utility’s point of view, if a Net ZEB is a heavy consumer in the winter, it will appear to be quite similar to a conventional building, requiring the use of additional generation. The increase in the generation-load match means reducing the s…